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Exploring the i-Ready Predictive Capability 


Curriculum Associates’ i-Ready is an adaptive diagnostic and individualized instructional tool 
that has been used in M-DCPS in the last few years. In addition, in 2016-2017, Curriculum 
Associates provided the District with results of their predictive model, which used the outcomes 
of the Fall and Winter i-Ready diagnostic testing to estimate the probabilities of a student scoring 
at every achievement level on the 2017 FSA ELA and Mathematics for students in grades 3-8. In 
this Brief, the results of the 2017 FSA are used to investigate the quality of the i-Ready’s 
predictive results. 


To enable a comparison of the i-Ready predictive results with the 2017 FSA observed results, the 
following procedure was used. The Winter i-Ready’s reported probabilities of scoring in each of 
the achievement levels 3-5 on the 2017 FSA were added, and the results were converted into a 
dichotomous variable coded as | if the probability of scoring within achievement levels 3-5 was 
at least 0.5, and 0 otherwise. In addition, achievement level on the 2017 FSA was dichotomized 
and coded as 1 if a student scored within achievement levels 3-5, and 0 if the student scored 
within achievement levels 1-2. 


The results of the comparison of the 2017 FSA ELA and the predicted Winter i-Ready results for 
Grade 3 students are shown below. 


i-Ready Prediction 


0 1 Total 
2017 0 8353 1512 9865 
FSA ELA 1 2267 10757 13024 
Total 10620 12269 22889 


It can be seen that of the 10620 Grade 3 students who were predicted to score within 
achievement levels 1-2 on the 2017 FSA ELA, 8353 in fact scored that way. The corresponding 
cell in the table above is generally referred to as containing the True Negative (TN) results. 
Similarly, 10757 students who were predicted to score within achievement levels 3-5 in fact 
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scored that way. These are generally referred to as True Positive (TP) results. 


One measure of predictive success is its Accuracy, defined as a percentage of correct predictions. 


Correct Predictions TN+TP _ 8353+10757 


For the table above, Accuracy= - eer = “Total = ~ <9aae. = 83% 


Because predicted and actual achievement levels may agree by chance, the accuracy results are 
generally accompanied by a statistical measure of agreement that corrects for a chance 
agreement. One such popular measure is Cohen’s Kappa. It achieves a maximum value of 1| if 
the predicted and actual results are identical. Values of the Kappa in the 0.61 — 0.80 range are 
often interpreted as indicators of a substantial agreement, while the values in the 0.41 — 0.60 
range as indicators of a moderate agreement. 


Below are the results of an analysis of the capability of the Winter i-Ready diagnostic results to 
predict the 2017 FSA ELA and Mathematics results by grade level and subject area. 


ELA Mathematics 
Grade | Accuracy Kappa | Accuracy Kappa 
3 83% 0.67 83% 0.64 
4 84% 0.67 84% 0.64 
5 84% 0.68 83% 0.65 
6 83% 0.67 84% 0.69 
U 83% 0.67 84% 0.68 
8 82% 0.63 79% 0.54 
Total 83% 0.67 83% 0.66 
Conclusion 


It can be observed that the results displayed in the table above demonstrate a high accuracy of 
the Winter i-Ready results for predicting the 2017 FSA outcomes in both subjects and across all 
grade levels shown. In addition, they show a substantial agreement between the predicted and 
actual outcomes in both subjects and across all grade levels, except for Grade 8 in mathematics. 
In that one case, the value of the Cohen’s Kappa falls within a moderate agreement range. 


Additional Measures of Predictive Success 


Predictive abilities of various tests are often reported using several additional measures. In the 
remainder of this brief, some of these measures are defined and presented. 


Looking back at the table on the first page, one can see that 2267 students in Grade 3 were 
predicted to score within achievement levels 1-2, but scored instead within achievement levels 3- 
5 on the 2017 FSA ELA. These cases are referred to as False Negatives (FN). Similarly, 1512 


students were predicted to score within achievement levels 3-5, but instead scored within 
achievement levels 1-2. These are referred to as False Positives (FP). 


These are four additional measures of a predictive success of a test: 


e Sensitivity is the percentage equivalent of the following fraction a In our context it 


answers the following question: of all students who scored within achievement levels 3-5, 
what percent were predicted to score that way? 

TN 
TN+FP 
achievement levels 1-2, what percent were predicted to score that way? 


e Positive Predictive Value (PPV) = ve 


predicted to score within achievement levels 3-5, what percent scored that way? 


e Negative Predictive Value (NPV) = cure 


predicted to score within achievement levels 1-2, what percent scored that way? 


e Specificity = is the answer to the following question: of all students who scored within 


answers the question: of all students who were 


answers the question: of all students who were 


The table below presents these additional indices. 


ELA Mathematics 
Grade Sensitivity Specificity PPV NPV _| Sensitivity Specificity PPV NPV 
3 83% 85% 88% 79% 87% 77% 87% 76% 
4 88% 79% 84% 83% 86% 80% 90% 73% 
5 86% 82% 85% 84% 80% 87% 90% 75% 
6 86% 81% 82% 85% 84% 85% 85% 83% 
Uf 85% 82% 83% 84% 84% 84% 82% 86% 
8 91% 71% 79% 87% 13% 82% 68% 85% 
Total 86% 80% 83% 83% 83% 83% 86% 80% 


Using these indices, one can answer other questions of interest. For instance, if one is interested 
in a situation where a student was predicted to score within achievement levels 3-5, but instead 
scored within achievement levels 1-2, a question that can be asked is, “Of all students who were 
predicted to score within achievement levels 3-5, what percent scored below that?” The answer 
is |-PPV. For Grade 3 students in ELA, the answer is 100% — 88% = 12%. 


Summary 

The results of the analyses presented above indicate that various indices of a test’s predictive 
success are sufficiently high with a possible exception of the values of the Cohen’s Kappa and 
PPV for Grade 8 students in mathematics. Therefore, the usage of the i-Ready diagnostic 
assessment as a predictive tool for the FSA outcomes in grades 3-8 is justified. 


